Modeling multimodal human-computer interaction

Multimodal Human Computer Interaction

Multimodal Human-computer Interaction

While human-to-human communication takes advantage of an abundance of information and cues, human-computer interaction is limited to only a few input modalities (usually only keyboard and mouse) and provides little flexibility as to choice of communication modality. In this paper, we present an overview of a family of research projects we are undertaking at Carnegie Mellon and Karlsruhe Univers...

Modeling of Modality Selection in Multimodal Human-Computer Interaction

Research Interests The main interest of my research is the development of model-based methods for simulation and automated usability prediction of multimodal interfaces. In particular I want to investigate how modality choice of users can be predicted and simulated by a computational model that estimates the quality of multimodal interfaces. Therefore I am also interested in exploring rules and...

Describing Multimodal Human-Computer Interaction

In this paper we argue that approaches to annotate multimodal human face-to-face interaction are not suitable for current device-based human-computer interaction. Instead, existing extensions proposed to established parameters describing the interaction with spoken dialog systems are presented and discussed. A standardization activity for annotating user interactions with multimodal systems is ...

Connectionist Models in Multimodal Human-computer Interaction

We present an overvi ew of our l aboratori es' research on Mul timodal Human-Computer Interf aces. By expl oi ti ng al l avai l abl e channel s of human communi cati on we aim to i ncrease exi bi l i ty, robustness, and natural ness of human-computer i nteracti on. The i nformati on sources we process i ncl ude Speech-, Character, and Gesture Recogni ti on, Faceand Eye Tracki ng, Li preadi ng, ...